Camera calibration apparatus

ABSTRACT

The invention provides a camera calibration apparatus which eliminates the necessity to effect mapping making a distinction among a plurality of characteristic points and can prevent complication in mapping processing even if the number of characteristic points increases. An object imaging section images a sphere whose magnitude and position in a three-dimensional coordinate system are known, and a magnitude/position detection section determines a magnitude and a position of the sphere on a screen from an image imaged by the object imaging section. A center position estimation section estimates a three-dimensional position of the center of the sphere from the magnitude and the position of the sphere on the screen determined by the magnitude/position detection section, and a parameter calculation section calculates a position of the object imaging section in the three-dimensional coordinate system based on the three-dimensional position of the center of the sphere estimated by the center position estimation section.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a camera calibration apparatus, andmore particularly to a camera calibration apparatus which estimates theposition and the direction of a camera from the positions ofcharacteristic points of an image imaged.

2. Description of the Related Art

In a conventional camera calibration apparatus of the type mentioned, anobject having three or more characteristic points (or characteristiclines) whose relative positions are known is imaged, and calibrationwherein the position and the direction of the camera are estimated fromthe positions of the characteristic points in the thus imaged image isperformed.

Conversely to the calibration described above, as an object postureestimation method, a tetrahedron arrangement estimation method whichuses four or more characteristic points, a three-line segment andtwo-point arrangement estimation method which uses three or more linesegments and two or more characteristic points, and so forth have beenproposed. While any of the object posture estimation methods estimatesthe posture of an object, since the relationship between the camera andthe object is relative, it is also possible to estimate the position orthe direction of the camera using the method.

The tetrahedron arrangement estimation method is disclosed in JapanesePatent Laid-Open Application No. Heisei 1-232484, and the three-linesegment and two-point arrangement estimation method is disclosed inJapanese Patent Laid-Open Application No. Heisei 2-51008.

In the conventional camera calibration described above, processing ofmapping between characteristic points originating from an image andcharacteristic points of an original object imaged is required. In thisinstance, in order to raise the accuracy in calibration, it is necessaryto increase the number of characteristic points. However, as the numberof characteristic points increases, the mapping processing becomes lessstable and more complicated.

Further, with the tetrahedron arrangement estimation method, thethree-line segment and two-point estimation method and so forth as anobject posture estimation method, it is difficult to stably extract fouror more characteristic points (characteristic lines) depending uponimaging conditions. As a result, even if an error occurs with themapping between characteristic points, no countermeasure to correct theerror can be taken.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a camera calibrationapparatus which eliminates the necessity to effect mapping making adistinction among a plurality of characteristic points and can preventcomplication in mapping processing even if the number of characteristicpoints increases.

In order to attain the object described above, according to an aspect ofthe present invention, there is provided a camera calibration apparatus,comprising object imaging means for imaging a sphere whose magnitude andposition in a three-dimensional coordinate system are known,magnitude/position detection means for determining a magnitude and aposition of the sphere on a screen from an image imaged by the objectimaging means, center position estimation means for estimating athree-dimensional position of the center of the sphere from themagnitude and the position of the sphere on the screen determined by themagnitude/position detection means, and means for calculating a positionof the object imaging means in the three-dimensional coordinate systembased on the three-dimensional position of the center of the sphereestimated by the center position estimation means.

In the camera calibration apparatus, a sphere whose magnitude andposition in a three-dimensional coordinate system are known is imaged,and from the thus imaged image, the magnitude and the position of thesphere on the screen are determined. Then, from the magnitude and theposition of the sphere on the screen thus determined, thethree-dimensional position of the center of the sphere is estimated, andthe position of the camera in the three-dimensional coordinate system iscalculated based on the thus estimated three-dimensional position of thecenter of the sphere. Consequently, the camera calibration apparatus isadvantageous in that the necessity to perform mapping of a plurality ofcharacteristic points making a distinction among them can be eliminatedand complication of mapping processing can be prevented even if thenumber of characteristic points is increased.

The camera calibration apparatus may further comprise zenith detectionmeans for determining a position of the zenith of the sphere on thescreen from the image imaged by the object imaging means, zenithposition estimation means for estimating a three-dimensional position ofthe zenith from the position of the zenith on the screen determined bythe zenith detection means and the three dimensional position of thecenter of the sphere, and means for calculating a direction of theobject imaging means in the three-dimensional coordinate system based onthe three-dimensional position of the zenith estimated by the zenithposition estimation means and the three-dimensional position of thecenter of the sphere estimated by the center position estimation means.

As an alternative, the camera calibration apparatus may further compriseequator characteristic point detection means for determining a positionof a characteristic point on the equator of the sphere on the screenfrom the image imaged by the object imaging means, orthogonal vectorcalculation means for determining, from the position of thecharacteristic point on the equator determined by the equatorcharacteristic point detection means and the three-dimensional positionof the center of the sphere, a line segment orthogonal to a straightline interconnecting the characteristic point on the equator and thecenter of the sphere, and means for calculating a direction of theobject imaging means in the three-dimensional coordinate system based onthe three-dimensional position of the zenith estimated based on the linesegment determined by the orthogonal vector calculation means and thethree-dimensional position of the center of the sphere estimated by thecenter position estimation means.

In each of the camera calibration apparatus which comprises theadditional elements described above, the position of a characteristicpoint at the zenith or on the equator of a sphere on the screen isdetermined from an imaged image, and from the thus determined positionof the characteristic at the zenith or on the equator on the screen andthe three-dimensional position of the center of the sphere, a linesegment orthogonal to a straight line interconnecting thethree-dimensional position of the zenith or the characteristic point onthe equator and the center of the sphere is estimated. Then, based onthe thus estimated line segment orthogonal to the straight lineinterconnecting the three-dimensional position of the zenith or thecharacteristic point on the equator and the center of the sphere and thethree-dimensional position of the center of the sphere, the position andthe direction of the camera in the three-dimensional coordinate systemare calculated. Consequently, the camera calibration apparatus isadvantageous in that the necessity to perform mapping of a plurality ofcharacteristic points making a distinction among them can be eliminatedand complication of mapping processing can be prevented even if thenumber of characteristic points is increased.

Alternatively, the camera calibration apparatus may further compriseequator characteristic point detection means for determining positionsof a plurality of characteristic points on the equator of the sphere onthe screen from the image imaged by the object imaging means, equatorplane calculation means for determining an equator plane of the equatorfrom the positions of the plurality of characteristic points on theequator determined by the equator characteristic point detection meansand the three-dimensional position of the center of the sphere, andmeans for calculating a direction of the object imaging means in thethree-dimensional coordinate system based on the three-dimensionalposition of the zenith estimated based on the equator plane determinedby the equator plane calculation means and the three-dimensionalposition of the center of the sphere estimated by the center positionestimation means.

According to another aspect of the present invention, there is provideda camera calibration apparatus, comprising object imaging means forimaging a plurality of spheres whose magnitudes and positions in athree-dimensional coordinate system are known, magnitude/positiondetection means for determining magnitudes and positions of theplurality of spheres on a screen from an image imaged by the objectimaging means, center position estimation means for estimatingthree-dimensional positions of the centers of the plurality of spheresfrom the magnitudes and the positions of the plurality of spheres on thescreen determined by the magnitude/position detection means, and meansfor calculating a position and a direction of the object imaging meansin the three-dimensional coordinate system based on thethree-dimensional positions of the centers of the plurality of spheresestimated by the center position estimation means.

In the camera calibration apparatus, a plurality of spheres whosemagnitudes and positions in a three-dimensional coordinate system areknown are imaged, and from a thus imaged image, magnitudes and positionsof the plurality of spheres on the screen are determined. Then, from themagnitudes and the positions of the plurality of spheres on the screenthus determined, three-dimensional positions of the centers of theplurality of spheres are estimated, and based on the three-dimensionalpositions of the centers of the plurality of spheres estimated, aposition and a direction of the object imaging means in thethree-dimensional coordinate system are calculated. Consequently, thecamera calibration apparatus is advantageous in that the necessity toperform mapping of a plurality of characteristic points making adistinction among them can be eliminated and complication of mappingprocessing can be prevented even if the number of characteristic pointsis increased.

The above and other objects, features and advantages of the presentinvention will become apparent from the following description and theappended claims, taken in conjunction with the accompanying drawings inwhich like parts or elements are denoted by like reference symbols.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a construction of a camera calibrationapparatus to which the present invention is applied;

FIG. 2 is a diagrammatic view illustrating operation of an objectimaging section of the camera calibration apparatus of FIG. 1;

FIG. 3 is a diagrammatic view illustrating operation of a centerposition estimation section of the camera calibration apparatus of FIG.1;

FIG. 4 is a diagrammatic view illustrating a camera coordinate systememployed in the camera calibration apparatus of FIG. 1;

FIG. 5 is a block diagram showing a construction of another cameracalibration apparatus to which the present invention is applied;

FIG. 6 is a block diagram showing a construction of a further cameracalibration apparatus to which the present invention is applied;

FIG. 7 is a block diagram showing a construction of a still furthercamera calibration apparatus to which the present invention is applied;

FIG. 8 is a block diagram showing a construction of a yet further cameracalibration apparatus to which the present invention is applied; and

FIG. 9 is a diagrammatic view showing connected spheres used in thecamera calibration apparatus shown in FIG. 8.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring first to FIG. 1, there is shown in block diagram aconstruction of a camera calibration apparatus to which the presentinvention is applied. The camera calibration apparatus shown includes anobject imaging section 1, a sphere-magnitude/position detection section(hereinafter referred to merely as detection section) 2, a centerposition estimation section 3, and a parameter calculation section 4.

The object imaging section 1 images a sphere not shown. Here, themagnitude and the position of the sphere are known, and the sphere isplaced in a three-dimensional coordinate system (hereinafter referred toas world coordinate system) set in advance. The detection section 2detects the top end, the bottom end, the left end and the right end ofan image of the sphere on a screen imaged by the object imaging section1 to determine the magnitude and the position of the sphere.

The center position estimation section 3 estimates the position of thecenter of the sphere in the camera coordinate system from the magnitudeand the position of the sphere determined by the detection section 2.The parameter calculation section 4 determines the position of thecamera in the world coordinate system from the position of the center ofthe sphere in the camera coordinate system estimated by the centerposition estimation section 3.

Referring now to FIGS. 1 to 4, in the camera calibration apparatusdescribed above, estimation of the three-dimensional position of acamera 6 in the world coordinate system a is performed. First, a sphere5 is imaged by the object imaging section 1 (refer to FIG. 2). It is tobe noted that such a countermeasure as to color a support for the sphere5 with a same color as that of the background is taken so that thesupport may not be detected as part of the sphere 5.

It is assumed here that the magnitude and the position of the sphere 5to be imaged in the world coordinate system a are known. Also it isassumed that internal parameters of the camera 6 used for imaging suchas the focal length, the lens distortion, the image center and the sizeof the imaging plane are all known.

The detection section 2 detects the top end, the bottom end, the leftend and the right end of an image 5 a of the sphere 5 on a camera screen6 a imaged by the camera 6 and determines the magnitude and the positionof the image 5 a of the sphere 5 on the camera screen 6 a.

The center position estimation section 3 estimates the position of thecenter of the sphere 5 in a camera coordinate system b from themagnitude and the position of the image 5 a of the sphere 5 on thecamera screen 6 a determined by the detection section 2.

The estimation of the position of the center of the sphere 5 by thecenter position estimation section 3 is performed in the followingmanner. In the following description, all the coordinate system appliedis the camera coordinate system XYZ (refer to FIG. 4). The radius of thesphere 5 in the camera coordinate system b is represented by R, thecoordinates of the center are represented by (X₀, Y₀, Z₀), and thecoordinates of the zenith are represented by (X₁, Y₁, Z₁). Further, theimaging plane L of the camera 6 is represented by Z=f (f is the focallength of the camera lens).

A plane which passes the origin 0 and extends perpendicularly to the XZplane and besides is tangential with the sphere 5 can be represented byX+kZ=0. It is to be noted that, since the distance between the plane andthe center of the sphere 5 is equal to the radius R,

k={X ₀ ×Z ₀ ±R[(X ₀)²+(Z ₀)² −R ²]^(½) }/[R ²−(Z ₀)²]

Here, it is assumed that

k′={X ₀ ×Z ₀ +R[(X ₀)²+(Z ₀)² −R ²]^(½) }/[R ²−(Z ₀)²]  (1)

k″={X ₀ ×Z ₀ −R[(X ₀)²+(Z ₀)² −R ²]^(½) }/[R ²−(Z ₀)²]  (2)

If, as shown in FIG. 3, an xy coordinate system whose origin is given bythe center of the lens is set on the camera screen 6 a and the xcoordinates of the intersecting points of tangential points betweentangential planes and the sphere 5 on the camera screen 6 a arerepresented by x′ and x″ and besides it is assumed that r_(x)=x″−x′,then $\begin{matrix}{r_{x} = \quad {x^{''} - x^{\prime}}} \\{= \quad {f\left( {k^{\prime} - k^{''}} \right)}} \\{= \quad {\left\{ {2{{fR}\left\lbrack {\left( X_{0} \right)^{2} + \left( Z_{0} \right)^{2} + R^{2}} \right\rbrack}^{1/2}} \right\}/}} \\{\quad \left\lbrack {R^{2} - \left( Z_{0} \right)^{2}} \right\rbrack}\end{matrix}$

Here, since X₀ is sufficiently smaller than Z₀, it can be consideredthat (X₀/Z₀)²=0. Consequently, $\begin{matrix}{r_{x} = \quad {2{fR}\left\{ {{\left\lbrack X_{0} \right)^{2}/\left( Z_{0} \right)^{2}} + 1 -} \right.}} \\{\left. \left. \quad {R^{2}/\left( Z_{0} \right)^{2}} \right\rbrack^{1/2} \right\}/\left\lbrack {{R^{2}/Z_{0}} - Z_{0}} \right\rbrack} \\{= \quad {2{fR}{\left\{ \left\lbrack {1 - {R^{2}/\left( Z_{0} \right)^{2}}} \right\rbrack^{1/2} \right\}/}}} \\{\quad \left\{ {- {Z_{0}\left\lbrack {1 - {R^{2}/\left( Z_{0} \right)^{2}}} \right\rbrack}} \right\}} \\{= \quad {{- 2}{{fR}/\left\lbrack {\left( Z_{0} \right)^{2} - R^{2}} \right\rbrack^{1/2}}}}\end{matrix}$

Consequently,

(r _(x))²[(Z ₀)² −R ²]=4f ² R ²

Thus, if this is solved with regard to Z₀, then

(Z ₀)²=[4f ² R ²+(r _(x))² R ²]/(r _(x))²

Z ₀ ={R[4f ²+(r _(x))²]^(½) }/r _(x)

is obtained.

X₀, Y₀ can be determined in the following manner. From the expressions(1) and (2) above, $\begin{matrix}{{x^{\prime} + x^{''}} = {- {f\left( {k^{\prime} + k^{''}} \right)}}} \\{= {{- f}\left\{ {\left( {2X_{0}Z_{0}} \right)/\left\lbrack {R^{2} - \left( Z_{0} \right)^{2}} \right\rbrack} \right.}}\end{matrix}$

is obtained. Consequently,

X ₀=(x′+x″)[(Z ₀)² −R ²]/2fZ ₀

Also Y₀ can be determined similarly as

Y ₀=(y′+y″)[(Z ₀)² −R ²]/2fZ ₀

If the position of the center of the sphere 5 in the camera coordinatesystem b is determined, then the position of the camera 6 in the worldcoordinate system a can be determined. However, unless components ofrotation are known, transform between the world coordinate system a andthe camera coordinate system b cannot be performed, and therefore, it isassumed here that the camera 6 exhibits no rotation.

In this instance, the parameter calculation section 4 can determine theposition of the camera 6 in the world coordinate system a as (−X₀−X_(c),−Y₀−Y_(c), −Z₀−Z_(c)) if the position of the center of the sphere 5 inthe world coordinate system a is (X_(c), Y_(c), Z_(c)).

FIG. 5 shows in block diagram another camera calibration apparatus towhich the present invention is applied. Referring to FIG. 5, the cameracalibration apparatus is a modification to and different from the cameracalibration apparatus described hereinabove with reference to FIG. 1 inthat it includes a parameter calculation section 9 in place of theparameter calculation section 4 and additionally includes a zenithdetection section 7 and a zenith position estimation section 8.

In the camera calibration apparatus of FIG. 5, the object imagingsection 1 images a sphere which has a characteristic point at the zeniththereof, and from the thus imaged image, estimation of rotation of thecamera 6 is performed in addition to estimation of the position of thecamera 6.

In the camera calibration apparatus, the zenith detection section 7detects the position of the zenith of the sphere 5 of the image imagedby the object imaging section 1. Using the position of the zenithdetected in this manner and the position of the center of the sphere 5determined by the center position estimation section 3, the zenithposition estimation section 8 estimates the position of the zenith ofthe sphere 5 in the camera coordinate system b.

The estimation of the position of the zenith of the sphere 5 isperformed in the following manner. First, the coordinates of the zenithof the sphere 5 are represented by (X₁, Y₁, Z₁), and the coordinates ofthe zenith on the camera screen 6 a are represented by (x₁, y₁, z₁)(refer to FIG. 3).

Since the point (X₁Y₁, Z₁) is a point on the sphere 5, the equation ofthe sphere 5 is satisfied. Consequently,

(X ₁ −X ₀)²+(Y ₁ −Y ₀)²+(Z ₁ −Z ₀)² =R ²

Since x₁=fX₁/Z₁ and y₁=fY₁/Z₁, by substituting them into the expressionabove,

[(x ₁ Z ₁ /f)−X ₀]²+[(y ₁ Z ₁ /f)−Y ₀]²+(Z ₁ −Z ₀)² =R ²

is obtained.

By solving this with regard to Z₁,

Z ₁ =f·{(x ₁ X ₀ +y ₁ Y ₀ +fZ ₀ +D ^(½))/[(x ₁)²+(y ₁)² +f ²]}  (3)

is obtained, where

D=R ²[(x ₁)²+(y ₁)² +f ²]−(x ₁ Y ₀ −y ₁ X ₀)²−(fX ₀ −x ₁ Z ₀)²−(fY ₀ −y₁ Z ₀)²

Further, since X₁=x₁Z₁/f and Y₁=y₁Z₁/f, also X₁ and Y₁ can be determinedimmediately in a similar manner as described above.

Since the inclination of the sphere 5 in the camera coordinate system bcan be determined from the position of the center and the position ofthe zenith of the sphere 5, parameters of the position and the directionof the camera 6 in the world coordinate system a can be determined inthe following manner by the parameter calculation section 9.

Where a rotation around the X axis is represented by φ and a rotationaround the Z axis is represented by ψ,

X ₁ =R cos φ sin ψ+X ₀

Y ₁ =R cos φ cos ψ+Y ₀

Z ₁ =R sin φ+Z ₀

and accordingly,

φ=sin⁻¹[(Z ₁ −Z ₀)/R]

ψ=sin⁻¹[(X ₁ −Z ₀)/R cos φ]

Once φ and ψ are determined, a matrix representing a rotation of thecamera 6 can be represented by ${\begin{bmatrix}1 & 0 & 0 \\0 & {\cos \quad \varphi} & {{- \sin}\quad \varphi} \\0 & {\sin \quad \varphi} & {\cos \quad \varphi}\end{bmatrix}\quad\begin{bmatrix}{\cos \quad \psi} & {{- \sin}\quad \psi} & 0 \\{\sin \quad \psi} & {\cos \quad \psi} & 0 \\0 & 0 & 1\end{bmatrix}} = \begin{bmatrix}{\cos \quad \psi} & {{- \sin}\quad \psi} & 0 \\{\cos \quad {\varphi \cdot \sin}\quad \psi} & {\cos \quad {\varphi \cdot \cos}\quad \psi} & {{- \sin}\quad \varphi} \\{\sin \quad {\varphi \cdot \sin}\quad \psi} & {{\sin \quad {\varphi \cdot \cos}\quad \psi}\quad} & {\cos \quad \varphi}\end{bmatrix}$

Once the rotation matrix is determined, the position of the camera 6 inthe world coordinate system a can be determined by ${\begin{bmatrix}{\cos \quad \psi} & {{- \sin}\quad \psi} & 0 \\{\cos \quad {\varphi \cdot \sin}\quad \psi} & {\cos \quad {\varphi \cdot \cos}\quad \psi} & {{- \sin}\quad \varphi} \\{\sin \quad {\varphi \cdot \sin}\quad \psi} & {{\sin \quad {\varphi \cdot \cos}\quad \psi}\quad} & {\cos \quad \varphi}\end{bmatrix}\quad\begin{bmatrix}{- X_{0}} \\{- Y_{0}} \\{- Z_{0}}\end{bmatrix}} - \begin{bmatrix}X_{C} \\Y_{C} \\Z_{C}\end{bmatrix}$

where (X_(c), Y_(c), Z_(c)) represent the position of the center of thesphere 5 in the world coordinate system a.

FIG. 6 shows in block diagram a construction of a further cameracalibration apparatus to which the present invention is applied.Referring to FIG. 6, the camera calibration apparatus is a modificationto and different from the camera calibration apparatus describedhereinabove with reference to FIG. 1 in that it includes a parametercalculation section 13 in place of the parameter calculation section 4and additionally includes an equator characteristic point detectionsection 10, an orthogonal vector calculation section 11 and a zenithposition estimation section 12.

In the camera calibration apparatus of FIG. 6, the object imagingsection 1 images a sphere 5 which has two characteristic points on theequator thereof, and from the imaged image, estimation of rotation of acamera 6 is performed in addition to estimation of the position of thecamera 6.

The equator characteristic point detection section 10 detects the twocharacteristic points located on the equator of the sphere 5 anddetermines the positions of the characteristic points on a camera screen6 a. It is to be noted that, although the two characteristic points neednot be distinguished from each other, for the convenience ofdescription, the characteristic points are denoted by M and N and thecoordinates of them are represented as M=(X₂, Y₂, Z₂) and N=(X₃, Y₃,Z₃), respectively. Further, the coordinates of them on the camera screen6 a determined by the equator characteristic point detection section 10are represented as (x₂, y₂) and (x₃, y₃), respectively.

Where the center of the sphere 5 is represented by C and the zenith isrepresented by T, the vector CT is a vector which is orthogonal to bothof the vectors CM and CN and can be determined in the following matter.First, the three-dimensional coordinates of the characteristic points Mand N can be determined by a quite same method as that by the zenithdetection section 7 described hereinabove.

In particular, for the characteristic point M, the coordinate Z₂ can bedetermined from

Z ₂ =f·{(x ₂ X ₀ +y ₂ Y ₀ +fZ ₀+(D ₂)^(½))/[(x ₂)²+(y ₂)² +f ²]}  (4)

where

D ₂ =R ²[(x ₂)²+(y ₂)² +f ²]−(x ₂ Y ₀ −y ₂ X ₀)²−(fX ₀ −x ₂ Z ₀)²−(fY ₀−y ₂ Z ₀)²

Meanwhile, for the characteristic point N, the coordinate Z₈ can bedetermined from

Z ₃ =f·{(x ₃ X ₀ +y ₃ Y ₀ +fZ ₀+(D ₃)^(½))/[(x ₃)²+(y ₃)² +f ²]}  (5)

where

D ₃ =R ²[(x ₃)²+(y ₃)² +f ²]−(x ₃ Y ₀ −y ₈ X ₀)²−(fX ₀ −x ₃ Z ₀)²−(fY ₀−y ₃ Z ₀)²

Further, since X₂=x₂Z₂/f, Y₂=y₂Z₂/f, X₃=x₃Z₃/f and Y₃=y₃Z₃/f, also thecoordinates X₂, Y₂, X₃ and Y₃ can be determined immediately in a similarmanner as described above.

Where the zenith is represented by (X₁, Y₁, Z₁), since the vector CT isorthogonal to both of the vectors CM and CN,

(X ₁ −X ₀) (X ₂ −X ₀)+(Y ₁ −Y ₀) (Y ₂ −Y ₀)+(Z ₁ −Z ₀) (Z ₂ −Z ₀)=0  (6)

(X ₁ −X ₀) (X ₃ −X ₀)+(Y ₁ −Y ₀) (Y ₃ Y ₀)+(Z ₁ −Z ₀) (Z ₃ Z ₀)=0  (7)

Further, since the zenith T is a point on the sphere 5,

(X ₁ −X ₀)²+(Y ₁ −Y ₀)²+(Z ₁ −Z ₀)² =R ²  (8)

and from the expression (6) above,

(Z ₁ −Z ₀)=−{[(X ₁ −X ₀) (X ₂ −X ₀)+−(Y ₁ −Y ₀) (Y₂ −Y ₀)]/(Z ₂ −Z₀)}  (9)

By substituting the expression (9) into the expression (7),

Y ₁ −Y ₀={[(Z ₂ −Z ₀)(X ₃ −X ₀)−(Z ₃ −Z ₀)(X ₂ −X ₀)]/[(Z ₃ −Z ₀) (Y ₂−Y ₀)−(Z ₂ −Z ₀₎ ₍ Y ₃ −Y ₀)]} (X ₁ −X ₀)

Here, if it is placed that

W=[(Z ₂ −Z ₀) (X ₃ −X ₀)−(Z ₃ −Z ₀)(X ₂ −X ₀)]/[(Z ₃ −Z ₀) (Y ₂ −Y ₀)−(Z₂ −Z ₀) (Y ₃ −Y ₀)]

then,

Y ₁ −Y ₀ =W(X ₁ −X ₀)  (10)

is obtained. Thus, by substituting the expressions (9) and (10) into theexpression (8) and arranging them,

(1+W ²) (X _(i)−X₀)²+{[(X ₁ −X ₀) (X ₂ −X ₀)+W(X ₁ −X ₀) (Y ₂ −Y ₀)]/(Z₂ −X ₀)}² =R ²

 (1+W ²) (X ₁ −X ₀)²(Z ₂ −X ₀)²+(X ₁ −X ₀)²[(X ₂ −X ₀)+W(Y ₂ −Y ₀)]² =R²(Z ₂ −X ₀)²

(X ₁ −X ₀)² [(1+W ²) (Z ₂ −X ₀)²+(X ₂ −X ₀)²+2W(X ₂ −X ₀) (Y ₂ −Y ₀)+W²(Y ₂ −Y ₀)² ]−R ²(Z ₂ −X ₀)²=0

is obtained.

Since this expression is a quadratic equation of X₁−X₀, X₁−X₀ can bedetermined by solving the expression. Further, Y₁−Y₀ and Z₁−Z₀ can bedetermined successively from the expressions (9) and (10).

Since the position (X₀, Y₀, Z₀) of the center of the sphere 5 has beendetermined by the center position estimation section 3, the zenithposition estimation section 8 can estimate the position (X₁, Y₁, Z₁) ofthe zenith.

FIG. 7 shows in block diagram a construction of a still further cameracalibration apparatus to which the present invention is applied.Referring to FIG. 7, the camera calibration apparatus is a modificationto and different from the camera calibration apparatus describedhereinabove with reference to FIG. 6 in that it includes a parametercalculation section 16 in place of the parameter calculation section 13and additionally includes an equator plane calculation section 14 and azenith position estimation section 15.

In the camera calibration apparatus of FIG. 7, the object imagingsection 1 images a sphere 5 having a plurality of characteristic pointson the equator thereof, and from the imaged image, estimation ofrotation of the camera 6 is performed in addition to estimation of theposition of the camera 6.

The equator plane calculation section 14 determines an equation of anequator plane of the sphere 5 in such a manner as described below by aleast square method using the characteristic points determined by theequator characteristic point detection section 10. In this instance,where the number of characteristic points detected is n, thethree-dimensional coordinates of them are represented by (X₀₁, Y₀₁,Z₀₁), (X₀₂, Y₀₂, Z₀₂), . . . , (X_(0n), Y_(0n), Z_(0n)), and thecoordinates of them on the camera screen 6 a are represented by (x₀₁,y₀₁), (x₀₂, y₀₂), . . . , (x_(0n), y_(0n)), respectively.

The positions (X_(0i)i, Y_(0i), Z_(0i)) of the characteristic points canbe determined in a manner quite similar as in estimation of the positionof the zenith. In particular,

Z _(0i) =f·{(x _(0i) X ₀ +y _(0i) Y ₀ +fZ ₀+(D _(i))^(½))/[(x _(0i))²+(y_(0i))² +f ²]}

X _(0i) =x _(0i) Z _(0i) /f

Y _(0i) =y _(0i) Z _(0i) /f

where $\begin{matrix}{D_{i} = \quad {{R^{2}\left\lbrack {\left( x_{0i} \right)^{2} + \left( y_{0i} \right)^{2} + f^{2}} \right\rbrack} - \left( {{x_{0i}Y_{0}} - {y_{0i}X_{0}}} \right)^{2} -}} \\{\quad {\left( {{fX}_{0} - {x_{0i}Z_{0}}} \right)^{2} - \left( {{fY}_{0} - {y_{0i}Z_{0}}} \right)^{2}}}\end{matrix}$

Once the three-dimensional coordinates of the characteristic points aredetermined, the equator plane can be determined in such a manner asdescribed below by a minimum square method. It is examined that, when ncoordinates (X_(0i), Y_(0i), Z_(0i)) (i=1, . . . , n) on the equator areobtained, an equation of the equator plane is placed as Z=aX+bY+c, andthe coefficients a, b and c in the equation are estimated. Since thecharacteristic points are positioned on the equator plane, the estimatedvalue Zy_(0i) of the coordinate Z_(0i) is given byZy_(0i)=aX_(0i)+bY_(0i)+c.

The coefficients a, b and c of the equation above are determined so thatthe sum S of the squares of the differences between the actualmeasurement values Z_(0i) and the estimated values Zy_(0i) may beminimized. Here, the sum S of the squares of the differences between theactual measurement values Z_(0i) and the estimated values Zy_(0i) isrepresented by $\begin{matrix}{S = {\sum\limits_{i = 1}^{n}\left( {Z_{0i} - {Zy}_{0i}} \right)^{2}}} \\{= {\sum\limits_{i = 1}^{n}\left\lbrack {Z_{0i} - \left( {{aX}_{0i} + {bY}_{0i} + c} \right)} \right\rbrack^{2}}}\end{matrix}$

In order to determine the coefficients a, b and c which minimize thesquare sum S, the simultaneous equations

{overscore (Z)}=a{overscore (X)}+b{overscore (Y)}+c

σ_(XZ) =a(σ _(X))² +bσ _(XY)

σ_(YZ) =aσ _(XY) +b(σ _(Y))²

should be solved.

where the average X, the variance (σ_(x))² and the covariance σ_(XY) aregiven respectively by

{overscore (X)}=(1/n)·Σ^(n) _(i=1) X _(0i)

(σ_(x))²=(1/n)·Σ^(n) _(i=1)(X _(0i) −{overscore (X)})²

σ_(XY)=(1/n)·Σ^(n) _(i=1)(X _(0i) −{overscore (X)}) (Y _(0i) −{overscore(Y)})

Solving the equations above,

a=[σ _(XZ)(σ_(Y))²−σ_(XY)σ_(YZ)]/[(σ_(X))²(σ_(Y))²−(σ_(XY))²]

b=[σ _(YZ)(σ_(X))²−σ_(XY)σ_(XZ)]/[(σ_(X))² (σ_(Y))²−(σ_(XY))²]

c={overscore (Z)}−a{overscore (X)}−b{overscore (Y)}

are obtained. Consequently, the equation Z=aX+bY+c is determined.

Since the position (X₀, Y₀, Z₀) of the center is estimated by the centerposition estimation section 3 and the direction vector (a, b, −1) of theaxis of rotation is estimated by the equator plane calculation section14, the position (X₁, Y₁, Z₁) of the zenith can be determined in thefollowing manner by the zenith position estimation section 15.

In particular, when

t=R(a ² +b ²+1)^(½)

the following expression is obtained

(X ₁, Y₁, Z₁)=(X ₀, Y₀, Z₀)+(a, b, −1)t

FIG. 8 shows in block diagram a construction of a yet further cameracalibration apparatus to which the present invention is applied, andFIG. 9 shows connected spheres used in the camera calibration apparatusof FIG. 8. Referring to FIGS. 8 and 9, the camera calibration apparatusincludes an object imaging section 1 similar to that describedhereinabove with reference to FIG. 1 and further includes a regiondivision section 17, a magnitude/position detection section 18, a centerposition estimation section 19 and a parameter calculation section 20.

In the camera calibration apparatus shown in FIG. 8, the object imagingsection 1 images an object like iron dumbbells composed of a pluralityof spheres connected to each other (refer to FIG. 9). It is assumed herethat the positions of the centers and the radii of the spheres A and B,which compose the object like dumbbells, in a world coordinate system aare known.

It is also assumed that the spheres A and B imaged by the object imagingsection 1 do not overlap with each other on the camera screen 6 a andthe connection portion between the spheres is colored in a same color asthat of the background or a like countermeasure is taken so that it maynot be detected as part of any of the spheres.

The region division section 17 performs region division so that imagesof the plurality of spheres A and B on the camera screen 6 a may bedistinguished from each other. The method of the region division may besuch that, for example, if the background color is black and the spheresA and B are white, then a pixel of white is searched first, and thenthose pixels around the white pixel which are white similarly should bedetermined that they belong the same region.

After the regions of the spheres A and B are divided successfully,processing of the magnitude/position detection section 18 and the centerposition estimation section 19 is performed for each of the regions in asimilar manner as in the camera calibration apparatus describedhereinabove with reference to FIG. 1 to estimate the positions of thecenters of the spheres A and B.

In the case of the example shown in FIG. 9 in which two spheres areinvolved, where the position of the center of the sphere A in the worldcoordinate system a is represented by (X₀, Y₀, Z₀) and the position ofthe center of the sphere B is represented by (X₁, Y₁, Z₁), cameraparameters can be estimated by processing by the parameter calculationsection 20 which is similar to the processing described hereinabove inconnection with the camera calibration apparatus of FIGS. 1 and 5 to 7.Further, if three or more spheres are involved, also a component ofrotation around the Y axis which is not determined in the cameracalibration apparatus described hereinabove can be determined.

In this manner, by using a sphere as an imaging object to be imaged bythe object imaging section 1 and using an image of the imaging objectitself imaged by the object imaging section 1 for calibration, thenumber of characteristic points of the imaging object can be reduced,and since a similar result can be obtained whichever ones ofcharacteristic points at the zenith and on the equator are used,stabilized characteristic point extraction processing which does notrely upon imaging conditions can be performed.

Further, also where the zenith of the sphere 5 is used, only onecharacteristic point is used. Furthermore, even where the equator of thesphere 5 is utilized, there is no need of performing mapping of aplurality of characteristic points distinguishing them from each other,and consequently, even if the number of characteristic points isincreased, the mapping processing is not complicated.

While preferred embodiments of the present invention have been describedusing specific terms, such description is for illustrative purposesonly, and it is to be understood that changes and variations may be madewithout departing from the spirit or scope of the following claims.

What is claimed is:
 1. A camera calibration apparatus, comprising:object imaging means for imaging a sphere whose magnitude and positionin a three-dimensional coordinate system are known; magnitude/positiondetection means for determining a magnitude and a position of the sphereon a screen from an image imaged by said object imaging means; centerposition estimation means for estimating a three-dimensional position ofthe center of the sphere from the magnitude and the position of thesphere on the screen determined by said magnitude/position detectionmeans; and means for calculating a position of said object imaging meansin the three-dimensional coordinate system based on thethree-dimensional position of the center of the sphere estimated by saidcenter position estimation means.
 2. A camera calibration apparatus asclaimed in claim 1, wherein the sphere has a characteristic point at azenith or/and the equator thereof.
 3. A camera calibration apparatus asclaimed in claim 1, further comprising: zenith detection means fordetermining a position of the zenith of the sphere on the screen fromthe image imaged by said object imaging means; zenith positionestimation means for estimating a three-dimensional position of thezenith from the position of the zenith on the screen determined by saidzenith detection means and the three dimensional position of the centerof the sphere; and means for calculating a direction of said objectimaging means in the three-dimensional coordinate system based on thethree-dimensional position of the zenith estimated by said zenithposition estimation means and the three-dimensional position of thecenter of the sphere estimated by said center position estimation means.4. A camera calibration apparatus comprising: object imagining means forimagining a sphere whose magnitude and position in a three-dimensionalcoordinate system are known; magnitude/position detection means fordetermining a magnitude and a position of the sphere on a screen from animage imaged by said object imaging means; center position estimationmeans for estimating a three-dimensional position of a center of thesphere from the magnitude and the position of the sphere on the screendetermined by said magnitude/position detection means; means forcalculating a position of said object imaging means in thethree-dimensional coordinate system based on the three-dimensionalposition of the center of the sphere estimated by said center positionestimation means; equator characteristic point detection means fordetermining a position of a characteristic point on an equator of thesphere on the screen from the image imaged by said object imaging means;orthogonal vector calculation means for determining, from the positionof the characteristic point on the equator determined by said equatorcharacteristic point detection means and the three-dimensional positionof the center of the sphere, a line segment orthogonal to a straightline interconnecting the characteristic point on the equator and thecenter of the sphere; and means for calculating a direction of saidobject imaging means in the three-dimensional coordinate system based onthe three-dimensional position of a zenith estimated based on the linesegment determined by said orthogonal vector calculation means and thethree dimensional position of the center of the sphere estimated by saidcenter position estimation means.
 5. A camera calibration apparatuscomprising: object imaging means for imaging a sphere whose magnitudeand position in a three-dimensional coordinate system are known;magnitude/position detection means for determining a magnitude and aposition of the sphere on a screen from an image imaged by said objectimaging means; center position estimation means for estimating athree-dimensional position of a center of the sphere from the magnitudeand the position of the sphere on the screen determined by saidmagnitude/position detection means; means for calculating a position ofsaid object imaging means in the three-dimensional coordinate systembased on the three-dimensional position of the center of the sphereestimated by said center position estimation means; equatorcharacteristic point detection means for determining positions of aplurality of characteristic points on the equator of the sphere on thescreen from the image imaged by said object imaging means; equator planecalculation means for determining an equator plane of the equator fromthe positions of the plurality of characteristic points on the equatordetermined by said equator characteristic point detection means and thethree-dimensional position of the center of the sphere; and means forcalculating a direction of said object imaging means in thethree-dimensional coordinate system based on the three-dimensionalposition of a zenith estimated based on the equator plane determined bysaid equator plane calculation means and the three-dimensional positionof the center of the sphere estimated by said center position estimationmeans.
 6. A camera calibration apparatus, comprising: object imagingmeans for imaging a plurality of spheres whose magnitudes and positionsin a three-dimensional coordinate system are known; magnitude/positiondetection means for determining magnitudes and positions of theplurality of spheres on a screen from an image imaged by said objectimaging means; center position estimation means for estimatingthree-dimensional positions of the centers of the plurality of spheresfrom the magnitudes and the positions of the plurality of spheres on thescreen determined by said magnitude/position detection means; and meansfor calculating a position and a direction of said object imaging meansin the three-dimensional coordinate system based on thethree-dimensional positions of the centers of the plurality of spheresestimated by said center position estimation means.